The present invention relates to an image processing method and apparatus for generating a mosaic image by combining a plurality of material images in a mosaic pattern, and a storage medium which stores a program for making a computer implementation of the method.
As is well known, mosaic is xe2x80x9ca design made by inlaying a combination of small pieces of various colors of stone, glass, marble, and the like into a floor, wall, or the like, or a technique of obtaining such designxe2x80x9d (Gendai Kokugo Jiten, Sanseido). Using this technique, a design or a picture image can be formed by combining a large number of picture images.
However, upon generating such mosaic image, when the number of material images (tile images) that form the mosaic image is small, the quality of the mosaic image using those material images deteriorates since the color and texture become different from those of an original image. For this reason, an image database that stores a larger number of material images is preferably prepared. However, in order to store many material images, a large-size storage is required. However, it is not practical since it results in an increase in cost.
Even when a large number of material images are prepared, if their colors are offset to specific colors, the generated image has a color different from that of an original image.
The present invention has been made in consideration of the aforementioned prior art, and has as its object to provide an image processing method and apparatus, which can generate a visually natural mosaic image by increasing the number of material images that form a mosaic image so as to improve the quality of the mosaic image generated using those images, and a storage medium.
It is another object of the present invention to provide an image processing method and apparatus, which generates many material images from a limited number of database material images, and generates a mosaic image using these material images.
It is still another object of the present invention to provide an image processing method and apparatus, which can reduce any color tone differences between an original image and mosaic image by executing color conversion to make colors of tile areas of the original image to be close to those of the material images.
It is still another object of the present invention to provide an image processing method and apparatus, which can reduce any color tone differences between an original image and mosaic image by executing color conversion to make colors of the material images to be close to those of tile areas of the original image.
The present invention presents its contents as a plurality of claims, each of which is made to achieve at least one of the above-mentioned objects.
More specifically, an image processing apparatus for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises storage means for storing a plurality of material images, material image generation means for generating another material image on the basis of a material image stored in the storage means, and mosaic image generation means for generating a mosaic image using a plurality of material images generated by the material image generation means.
An image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises the material image generation step of generating another material image on the basis of each of a plurality of stored material images, and the mosaic image generation step of generating a mosaic image using a plurality of material images generated in the material image generation step.
A computer-readable storage medium, which stores a program for implementing an image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises a material image generation step module for generating another material image on the basis of each of a plurality of stored material images, and a mosaic image generation step module for generating a mosaic image using a plurality of material images generated in the material image generation step module.
An image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises the step of adding a still image captured from a moving image at a predetermined timing to material images, and the mosaic image generation step of generating a mosaic image on the basis of the added material images.
An image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises the material image addition step of adding a still image captured from a moving image at a predetermined timing to material images, the first feature amount calculation step of calculating a first feature amount in units of segmented images of an original image, the second feature amount calculation step of calculating a second feature amount of the added material image, the material selection evaluation step of calculating a material selection evaluation value using the first feature amount in units of segmented images and the second feature amount of the material image, and the optimal image selection step of selecting an optimal image on the basis of the material selection evaluation value.
An image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises the step of calculating first average luminance values of red, green, and blue tristimulus value components in units of segmented images of an original image, the step of calculating second average luminance values of red, green, and blue tristimulus value components of a plurality of material images, the material selection evaluation step of calculating a material selection evaluation value on the basis of the first and second average luminance values, and the optimal image selection step of selecting an optimal material image on the basis of the material selection evaluation value.
A computer-readable storage medium, which stores a program for implementing an image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises a step module for adding a still image captured from a moving image at a predetermined timing to material images, and a mosaic image generation step module for generating a mosaic image on the basis of the added material images.
A computer-readable storage medium, which stores a program for implementing an image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises a material image addition step module for adding a still image captured from a moving image at a predetermined timing to material images, a first feature amount calculation step module for calculating a first feature amount in units of segmented images of an original image, a second feature amount calculation step module for calculating a second feature amount of the added material image, a material selection evaluation step module for calculating a material selection evaluation value using the first feature amount in units of segmented images and the second feature amount of the material image, and an optimal image selection step module for selecting an optimal image on the basis of the material selection evaluation value.
A computer-readable storage medium, which stores a program for implementing an image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises a step module for calculating first average luminance values of red, green, and blue tristimulus value components in units of segmented images of an original image, a step module for calculating second average luminance values of red, green, and blue tristimulus value components of a plurality of material images, a material selection evaluation step module for calculating a material selection evaluation value on the basis of the first and second average luminance values, and an optimal image selection step module for selecting an optimal material image on the basis of the material selection evaluation value.
An image processing apparatus for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises arithmetic means for segmenting an original image into a plurality of areas, and calculating a first color distribution parameter on the basis of an average pixel value of each of the plurality of areas, calculation means for calculating a second color distribution parameter on the basis of an average pixel value of each of a plurality of material images, color conversion parameter calculation means for calculating a color conversion parameter for changing colors of the respective areas on the basis of the first and second color distribution parameters, color conversion means for converting the colors of the respective areas using the color conversion parameter calculated by the color conversion parameter calculation means, and mosaic image generation means for generating a mosaic image by selecting a material image corresponding to each area converted by the color conversion means.
An image processing apparatus comprises generation means for generating a plurality of derivative material images from each of a plurality of material images prepared in advance, segmentation means for segmenting an image to be processed into a plurality of blocks to obtain a plurality of block images, and replacement means for replacing each of the plurality of block images by the derivative material image on the basis of a feature amount of each of the plurality of block images and a feature amount of the derivative image generated by the generation means.
An image processing method comprises the generation step of generating a plurality of derivative material images from each of a plurality of material images prepared in advance, the segmentation step of segmenting an image to be processed into a plurality of blocks to obtain a plurality of block images, and the replacement step of replacing each of the plurality of block images by the derivative material image on the basis of a feature amount of each of the plurality of block images and a feature amount of the derivative image generated in the generation step.
A computer-readable storage medium, which stores a control program for making a computer implement image processing for generating a mosaic image, the control program comprises a generation step module for generating a plurality of derivative material images from each of a plurality of material images prepared in advance, a segmentation step module for segmenting an image to be processed into a plurality of blocks to obtain a plurality of block images, and a replacement step module for replacing each of the plurality of block images by the derivative material image on the basis of a feature amount of each of the plurality of block images and a feature amount of the derivative image generated by the generation step module.
According to a preferred aspect of the present invention, in the image processing apparatus, the material image generation means generates a material image by horizontally flipping a first material image.
According to a preferred aspect of the present invention, in the image processing apparatus, the material image generation means generates a material image having pixels obtained by converting color values of pixels of a first material image.
According to a preferred aspect of the present invention, in the image processing apparatus, the material image generation means generates a material image by negative/positive-converting a first material image.
According to a preferred aspect of the present invention, in the image processing apparatus, the material image generation means generates a material image having pixels obtained by converting specific color values of pixels of a first material image.
According to a preferred aspect of the present invention, in the image processing method, the material image generation step includes the step of generating a material image by horizontally flipping a first material image.
According to a preferred aspect of the present invention, in the image processing method, the material image generation step includes the step of generating a material image having pixels obtained by converting color values of pixels of a first material image.
According to a preferred aspect of the present invention, in the image processing method, the material image generation step includes the step of generating a material image by negative/positive-converting a first material image.
According to a preferred aspect of the present invention, in the image processing method, the material image generation step includes the step of generating a material image having pixels obtained by converting specific color values of pixels of a first material image.
According to a preferred aspect of the present invention, in the storage medium, the material image generation step module generates a material image by horizontally flipping a first material image.
According to a preferred aspect of the present invention, in the storage medium, the material image generation step module generates a material image having pixels obtained by converting color values of pixels of a first material image.
According to a preferred aspect of the present invention, in the storage medium, the material image generation step module generates a material image by negative/positive-converting a first material image.
According to a preferred aspect of the present invention, in the storage medium, the material image generation step module generates a material image having pixels obtained by converting specific color values of pixels of a first material image.
According to a preferred aspect of the present invention, the image processing method further comprises the non-processed material image list generation step of generating a non-processed material image list used for identifying material images which are newly added and are not used in a calculation of the material selection evaluation value from material images already used in the calculation of the material selection evaluation value, and the material selection evaluation value is calculated for only the unused material image.
According to a preferred aspect of the present invention, the image processing method further comprises the re-processing tile list generation step of generating a re-processing tile list for recording information that identifies if selection of the material image is unsettled, and the error reference discrimination step of discriminating a relationship between the calculated material selection evaluation value and a predetermined error reference value, and when the calculated material selection evaluation value is smaller than the predetermined error reference value, selection is settled as an optimal material image, and the material selection evaluation value is calculated for only areas where selection of the material image is unsettled.
According to a preferred aspect of the present invention, the image processing method further comprises the material image deletion step of deleting material image data which is not selected based on the material image selection evaluation value.
According to a preferred aspect of the present invention, in the image processing method, the optimal image selection step includes the step of selecting a material image which yields a minimum sum of squares of differences between the first and second feature amounts as an optimal material image.
According to a preferred aspect of the present invention, in the image processing method, the plurality of material images are added by capturing a moving image.
According to a preferred aspect of the present invention, in the image processing apparatus, the arithmetic means calculates the first color distribution parameter on the basis of R, G, and B values of pixels of each area.
According to a preferred aspect of the present invention, in the image processing apparatus, the calculation means calculates the second color distribution parameter on the basis of R, G, and B values of pixels of each material image.
According to a preferred aspect of the present invention, in the image processing apparatus, the arithmetic means calculates the first color distribution parameter on the basis of a* and b* values in an L*a*b* space of pixels of each area.
According to a preferred aspect of the present invention, in the image processing apparatus, the calculation means calculates the second color distribution parameter on the basis of a* and b* values in an L*a*b* space of pixels of each material image.
According to a preferred aspect of the present invention, in the image processing apparatus, the mosaic image generation means selects a material image which has a minimum distance between R, G, and B stimulus values of each area and R, G, and B stimulus values of each material image.
According to a preferred aspect of the present invention, in the image processing apparatus, the mosaic image generation means selects a material image which has a minimum distance between a* and b* values of each area and a* and b* values of each material image.
An image processing method for generating a mosaic image by combining a plurality of material images in a mosaic pattern, comprises the arithmetic step of segmenting an original image into a plurality of areas, and calculating a first color distribution parameter on the basis of an average pixel value of each of the plurality of areas, the calculation step of calculating a second color distribution parameter on the basis of an average pixel value of each of a plurality of material images, the color conversion parameter calculation step of calculating a color conversion parameter for changing colors of the respective areas on the basis of the first and second color distribution parameters, the color conversion step of converting the colors of the respective areas using the color conversion parameter calculated in the color conversion parameter calculation step, and the mosaic image generation step of generating a mosaic image by selecting a material image corresponding to each area converted in the color conversion step.
According to a preferred aspect of the present invention, in the image processing method, the arithmetic step includes the step of calculating the first color distribution parameter on the basis of R, G, and B values of pixels of each area.
According to a preferred aspect of the present invention, in the image processing method, the calculation step includes the step of calculating the second color distribution parameter on the basis of R, G, and B values of pixels of each material image.
According to a preferred aspect of the present invention, in the image processing method, the arithmetic step includes the step of calculating the first color distribution parameter on the basis of a* and b* values in an L*a*b* space of pixels of each area.
According to a preferred aspect of the present invention, in the image processing method, the calculation step includes the step of calculating the second color distribution parameter on the basis of a* and b* values in an L*a*b* space of pixels of each material image.
According to a preferred aspect of the present invention, in the image processing method, the mosaic image generation step includes the step of selecting a material image which has a minimum distance between R, G, and B stimulus values of each area and R, G, and B stimulus values of each material image.
According to a preferred aspect of the present invention, in the image processing method, the mosaic image generation step includes the step of selecting a material image which has a minimum distance between a* and b* values of each area and a* and b* values of each material image.
According to a preferred aspect of the present invention, in the image processing apparatus, the generation means generates the derivative material image by extracting a partial image from each of the plurality of material images.
According to a preferred aspect of the present invention, in the image processing apparatus, the generation means generates the derivative material images by extracting a plurality of partial images from each of the material images using a plurality of extraction templates.
According to a preferred aspect of the present invention, in the image processing apparatus, the replacement means replaces each of the plurality of block images by one of the derivative image and material image on the basis of the feature amount of each of the plurality of block images, and a feature amount of each of the plurality of derivative material images and the plurality of material images.
According to a preferred aspect of the present invention, the image processing apparatus further comprises enlargement/reduction means for executing enlargement/reduction to match image sizes of the derivative material image and material image with each other.
According to a preferred aspect of the present invention, in the image processing apparatus, the generation means comprises presentation means for presenting an extraction state of a derivative material image upon extracting the derivative material image from the material image, inquiry means for inquiring as to whether or not the extraction presented by the presentation means is to proceed, and deletion means for deleting the extracted derivative image when the inquiry means instructs that the extraction is not to proceed.
According to a preferred aspect of the present invention, in the image processing apparatus, the generation means recognizes an object in a partial image upon extracting the partial image from each of the plurality of material images, and extracts the partial image so as not to divide the recognized object.
According to a preferred aspect of the present invention, in the image processing apparatus, the generation means generates a table for storing feature amounts of the plurality of derivative material images obtained by extracting partial images from the plurality of material images prepared in advance in correspondence with derivation information that indicates a derivation method of the derivative material image, and the replacement means selects a derivative image used for each of the block images on the basis of the feature amount of each of the plurality of block images and the feature amount stored in the table, derives the selected derivative image on the basis of the derivation information stored in the table, and replaces each block image by the derived material image.
According to a preferred aspect of the present invention, in the image processing method, the generation step includes the step of generating the derivative material image by extracting a partial image from each of the plurality of material images.
According to a preferred aspect of the present invention, in the image processing method, the generation step includes the step of generating the derivative material images by extracting a plurality of partial images from each of the material images using a plurality of extraction templates.
According to a preferred aspect of the present invention, in the image processing method, the replacement step includes the step of replacing each of the plurality of block images by one of the derivative image and material image on the basis of the feature amount of each of the plurality of block images, and a feature amount of each of the plurality of derivative material images and the plurality of material images.
According to a preferred aspect of the present invention, the image processing method further comprises the enlargement/reduction step of executing enlargement/reduction to match image sizes of the derivative material image and material image with each other.
According to a preferred aspect of the present invention, in the image processing method, the generation step comprises the presentation step of presenting an extraction state of a derivative material image upon extracting the derivative material image from the material image, the inquiry step of inquiring as to whether or not the extraction presented in the presentation step is to proceed, and the deletion step of deleting the extracted derivative image when it is instructed in the inquiry step that the extraction is not to proceed.
According to a preferred aspect of the present invention, in the image processing method, the generation step includes the step of recognizing an object in a partial image upon extracting the partial image from each of the plurality of material images, and extracting the partial image so as not to divide the recognized object.
According to a preferred aspect of the present invention, in the image processing method, the generation step includes the step of generating a table for storing feature amounts of the plurality of derivative material images obtained by extracting partial images from the plurality of material images prepared in advance in correspondence with derivation information that indicates a derivation method of the derivative material image, and the replacement step includes the step of selecting a derivative image used for each of the block images on the basis of the feature amount of each of the plurality of block images and the feature amount stored in the table, deriving the selected derivative image on the basis of the derivation information stored in the table, and replacing each block image by the derived material image.